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ABSTRACT 

This report summarizes the results of three 
studies— the Regression studies which investigated factors that are 
at least partly under the control of school personnel, the Outlier 
Study which used multiple regression analysis to identify schools 
that were performing either above or below their predicted levels of 
achievement as computed from nonschool factors^ and the Observational 
Study which involved observation of ^^ schools that were identified 
as above or below predicted achievement in the Outlier Study. 
Emphasis is given to the relationship between student achievement and 
school size^ eipenditcres^ special programs^ traditional versus open 
education^ teacher characteristics, school conditions, and staff 
attitudes. Summaries of factors associated with high and low 
achievement are presented along with factors showing ambiguous 
relationships with achievement and factors not associated with 
achievement. The most encouraging findings relate to teacher 
characteristics, staff attitudes, and school practices. The 
relationships described are not necessarily causal. (Author/IRT) 
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FOREWORD 



The Bureau of School Programs Evaluation o£ the New York State^ 
Education Department has for a number of years been developing pro- 
cedures for evaluating the performance of schools and school districts. 
The Quality Measurement Project', running trom the late 1950* s to the 
mid-l960's, and the current Performance Indicators in Education Project 
were both developed to provide districts with more appropriate procedures 
for asse:>slng their influence on student achievement. 



Paralleling this interest has been an interest in what processes 
within schools influence achievement. To summarize research findings on 
this topic, a review was made of 97 studies dealing with factors related 
to student performance*. Two reports resulted: one, a detailed description 
of methods and results;^ the other, a suranary which related the findings 
to ten questions about the effectiveness of education. 



With this background, the bureau launched its own studies of school 
processes to learn more about what schools can do to Improve the achieve- 
ment of students. Three strategies were used. One strategy, de»ig«ata4 

here as the Regression Studies, Involved analyzing data in the Department's 
dita files to determine how various school district factors relate to 
achievement. The second strategy, the Outlier Study, involved identifying 
high- and low-performing school buildings and analyzing school data to find 
variables which distinguished between the two groups. The third strategy, 
the Observational Study, called for observing in high- and low-performing 
schools to discover classroom processes which differentiated between the 
contrasting groups of schools. The studies had a common purpose: to 
identify relationships between school factors and student performance. 

The present report sumnarizes the findings of the three sets of studies. 
An expanded edition, Three Strategies for Studying the Effects of School 
Processes, provides more detail on the procedures used in the studies. It 
13 available from the Bureau of School Programs Evaluation. 

A number of individuals contributed to the findings described here. 
The Outlier Study was carried out by Austin D. Swanson, Professc* of Educa- 
tional Administration, and Robert C Nichols, Professor of Educational 
Psychology, both of die University of New York at Buffalo. The Observational 
Study was directed by Ri -hard M. Clark, Professor of Educational Psychology of 
the University of New York at Albany ."^ The Regression Studies were carried out 
by the staif o£ the Bureau of §choal Progr-ms Evaluation: David J. Irvine, 
Chief; Gerald H. Wohlferd- Guy D. Spath; and Philip J. Pillsworth. 
Gennaro DiGiovanni, who served as a public administration intern during the 
time these studief were under way, conducted several of the regression 
analyses. Mr. Spath coordinated the consolidation of the several studies 
into this report and the expanded edition. 



John W. Polley 

Associate Comnissloner for 

Research, Planning and Evaluation 
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INTRODUCTION 



Public interest in education has incr*.*«.H i 

success. D.s.nj.xr:;:.":,i3?:: iz^i ^i^- 
J::: l^^i:-]^;^ oT^.r^r^" '--re. 

StlU Other factors may be important to the quality of life irrH.^; 
tional institutions Although they have no ann.r* f * I 
performance For i:hese r.-.n k -PPa^nt impact on student 

.xanlned for posslWe effect, on other a.pect. of student functioning, 
for Studvin, Kff>ct. nf ..Kool Pro...„. Three Str.tesie. 
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OVERVIEW 



Three ttts of studies of ichool factori w«re conducted under the 
aegis of ihe Bureau of School Prograot Evaluation. All of the studies 
had the purpose of identifying school factors which relate to the achieve- 
aent of etudents. They are reported In detail In Three Strategiee for 
Studying the Effects of School Processes . The present report describee 
briefly the three sets of studies and sunniarizee the findings. 

The studies reported here were designed to complement an earlier 
bureau publication, Wh^t Research Says About Impr ovlne Student Perform 
ounce . That publication suninarized the results of 97 research studies 
dealing with the relation of a vArtety ^f school factors to school Ottt- 
comes, whereas the present report attempts to integrate flndinga of 
research done more recently in the schools o£ New York State. 

It Is hoped that the results reported here can provide a basis 
for thoughtfuT discussion and, together with other infortnatlon, iuggest 
directions which can be pursued to Improve education. B^lta very 
nature, the research tloes not offer simple answera to the complex quee- 
tlons confronting education today. But If the flndinga seem to be 
cautiously Interpreted, perhaps that fact will help us avoid rushing 
to conclusions or Jumping on bandwagons. 



THE STUDIES 



RcRresslon Studies 

The first set of studies,, carried out by the bureau staff, inves* 
tlgated factors which are at least partly under the control of school 
personnel. As In the Coleman Report, the effects of nonschool factors 
were controlled ail the unique contributions of school variables were 
studied. The stualc:<» are referred to throughout the report aa the 
Rfgresslon Studies . 



The Outlier Study / 

The second type of study used multiple regression analysis to 
identify schools which were performing either above or below their pre- 
dicted levels of achievement, as computed from nonschool factors. The 
study focused on school buildings rather than school districts. Three 
groups of schools were identified: I) High outliers, those schools 
whose actual mean achievement scores were well above their predicted 
scores; 2) Low outliers, those schools whose actual acores were well 
below their predicted scores; sad 3) Mldll^rs, those schools whose 
actual scores fell near their predicted scores. These three groups of 
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•chooH were cowpared through analysH of variance in an ^.Itenpt to 
determine the effects of a number of variables whith reflect school 
procetsei. Because thii study omphasi/ed schools which lay some dis- 
tance from their expected levels of achievement, it was dubbed the 
QutPer Study . o 

The ObsTyational Study 

The third kind of study involved observing* In U schools which 
were identified as above or below predicted achievement/ in the 'Outlier 
Study described above. Observational Instruments, interviews, .nd 
qufcUionnaires were used to obtain tinformatlon about classroom Activ- 
ities interactions among students and staff members, and -perceptions 
of «taff members. This study is referred to as tha Qbservaf ional Study - 



FINDINCS 

Wliile each of the sets of studies described above used a some- 
what different approach, their findings «re consolidated below in an 
attempt to develop a coherent, though not necessarily complete, picture 
of how school processes relate to school outcomes, where the result,s 
are contradictory or ambiguous, an attempt is made to show this. 

t 

Is Size a Factor? 

The average school district enrollment in New York State in 1971 
and 1972 was approxintely 2500. ranging from 30 Lo over 30,000. in 
the Regression Studies, district enrollment was found to be negatively 
related to achievement; that is, larger districts had poorer average 
achievement. However, when total population of districts was considered 
it replaced enrollment as a predictor. This seems to suggest that the 
negative relationship between enrollment and achievement is a function 
of urbanness rather than of school size. The finding, then, does not 
appear to offer evidence about the optimum size of school* or school 
districts. 

The Outlier Study showed no difference in size between high and 
low outlier schools; each group averaged about 100 fewer students than 
did schools identified as midliers. In addition, classes tended to be 
smaller for both positive and negative outliers than for midliers. As ° 
has been suggested in other studies, the effect of class size may be 
dependent on the type of student and on the subject being taught. 
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po E^VndlCurca Affect Learning? 

• Several typea of expendlturea ware inveatigated . Slnct the 
largest aingle expenditure for education ia for aalariei, teachera' 
taUries wer^ studied in several different waya. 

*' * * . 
The RegressioniStudiea revealed a poaltiv^s relationihip between 

<,mfidian teacher aalaries of achool diatricta end average atudant achieve- 
ment in the diatricts. Theae findinga were Supported by the Outliera 
Study, in which teachers in achoola achieving a^ve expected earned the 
mo^t money and teechera in tchoola achieving below expected received . 
the least. While t'heae findings do not mean that higher aalaries 
produce higher performance, they indicate that the more successful dis- 
tricts do pay higher salariea. Not unexpectedly, salary- related variables, 
such as graduate credits and experience of teachers, were alao 
Shown to be related to achievement. It Is interesting to note that an 
earlier atudy. uaing 1969 data, showed that the amount of money spent 
per pupil on principals' salaries was also related to achievement. 

The Regreasion Studies elao examined full tax value and several 
district per-pupll e\penditures. including total expenditurea and expen- 
ditures for regular day Instruction, for teachers, for central admlnla- 
tretlon. and for principala. No consistent relationships were found. 
In apite of the fact that wide variations were observed from district 
to district. 



How Are Special Programs and 

Services Related to Achievement? 

Special programs frequently showed negative relationahlps with 
achiever* nt. The Outlier Study, for example, revealed a negative rela- 
tionahip between achievement and special, programs for the handicapped. 
Guidance, social work services, and attendance services were also nega- 
ti^^ely related to achievement, while programs for the academically 
talented were more llltely to be found In tne high outlier schoola. Theae 
findings c«n be misleading if cause and effect are assumed. Subprograms 
are most likely to be found where they are moat needed. Therefore. It 
Is not surprising to find that more special programs are found where 
achievement Is low. In .addition, the data available for these studies 
did not clearly distinguish between types of special programs; It Is 
likely that some types of programs affect students differently from 
other type^» of programs. 

In the Observational Study, hlgh-achlevlng SchooU were rated as 
having higher total activity in nine of eleven reading actlvltlea, <^ 
especially in silent reading, than did low-achlevlng schools. The Out- 
lier Study showed that the Mse of rooms for academic rather than voca- 
tional study also correlated positively with achievement. The Regreasion 
Studies Indicated a positive relation between attendance r«te and achieve- 
ment, but this relationi.bip disappeared when socioeconomic faftors wer* 
considered. The same phenomenon occurred in a study of student mobility. 
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Tr^dltlonal^vy. 

Attempts to move jway trom tracixtional teacher-centered class-' 
. room arrangements J^we included many innovations desij^ned to increase 
learning. The Outlier Study showed that most classrooms in the school, 
studied were traditional, with the smallest number of traditional 
(;l3S3rooms being found in .ow outlier schools and the greatest nutcber 
in midlier schools. High outUers were more likely to have open class- 
rooms and multi-age groupings. Multi-unit ^groupings w,-re negatively 
related CO performance. Midlier schools ^ere less likely to have / 
innovative programs than either of the other two groups of schools 
Other organizational arrangements were not significantly related to 
achievement-. These included clustering, continuous progress, depart- 
mentalisation differentiated staffing, dual progress plan, house plan, 
modular scheduling, non-graded, self-contained, and team teaching. The 
Observatio.,al .§tudy supported the findings of the Outlier Stady in 
respect to open classrooms. ^ 

Do Teach er Characteristics Make a Piffergnc^ ? 

When candidates for teaching positions walk mto a school superin- 
tendent s off ice. they bring with them certain personal and professional 
characteristics. Among these are their professional training and 
experience, age. sex. marital status, and a variety of personal traits. 
The superintendent may have very little concrete evidence to luse in 
selecting among candidates with an almost infinite number of Icombinat ion. 
of profassional and personal characteristics. Does a candidate with a 
doctorate have more t^ offer than one with many years of experience but 
less in the way af formal education? Are women more effective than men 
in elementary schools? How important is graduate school training when 
the teacher is expected to teach reading to eight -year-old children? 

The New York State Education Department collects data on teachers 
in each of the school districts in the state. From the available data, 
five teacher characteristic variables were selected for study: 

1. Median Age of Teachers in the District 

2. Median Years of Experience 
X. Percent of Married Teachers 
4. Percent of Male Teachers 

5 Percent of Teachers Having Graduate Credit 

The Regression Studies revealed no relationships between teacher 
age or experience and average student achievement m a district. The 
Outliers Study, on th- other hand, showed a positive relationship 
between teacher experience and student performance. Furthermore, a 
larger pe,:centage of the teachers in the high outlier schools were on 
tenure. 
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- The' percent of teachers having some graduate training showed the 
most consistent relationship with reading and nuthematics achitvement 
in the Regression Studies. Even after controlling for socioeconomic 
factors percent of teachers having 30 or more credit hours beyond Che 
bachelor's degree, demonstrated a significant relationship to all of the» 
4Chievemerft criteria. 
• ♦ ' 

^Significant shifts in teacher certification took place from 1970 ^ 
fo 1971. In 1970 only 230 districts had all teachers certified while 
In I97l*this number increaaed to over 480. The ,Regreasion Studies of 
this variable showed a definite negative relationship of percent uncerti* 
fied to student achievement in 1970. The tremendous reduction in number 
of schools having uncertified teachers in 1971 led to lesa conclusive 
results for that year, but the inferenc^^still aeems war ranted^ fhat- 
certification is desirable. . 

The positive -findings on certification and graduate training were 
both supported in the school-building analyses carried out in the Outliera 
Study. 

Using regression analysis to study school buildings, it was found 
that schools with a gre*»ter percentage of Mack teachers had higher levels 
of perrorraance. after controlling for non-sfchool factors. 

The findings on sex and marital status were mixed. In 1971, no 
relationship was found between percent of married teachers in a district 
and achievement. In 1972. a positive relationship was evident. Percent 
of male teachera was negatively related to all achievement criteria in 
1972 but only to sixth-^rade mathematics in 1971. 
& . 

While these results suggest that Relationships exist be- 
tween student achievement and teacher •graduate trainingi it should be 
noted that theae relationships may reflect other factors related to both 
achievement and teacher characteristics. For instance, we know that 
low socioeconomic school districts tend to have low mean scorea on achieve 
ment tests and high socioeconomic districts tend to have high mean acores. 
The positive relationship that seems to exist betvTeen percent of teichers 
with graduate credits and student achievement may simply reflect a ten- 
dency among higher socioeconomic districts to employ teachera with gradu- 
ate training. Conversely, the lower socioeconomic districts may not 
have the money to pay the higher salaries of teschers with graduate 
credits. 

1 i 

The hypothesis thai these teacher characteristic ' lonships 
are merely reflective of the known relationship between socioeconomic 
status and achievement Is even more tenable regarding teacher marital 
status and sex. Lower socioeconomic districts might be expected to 
hire more men since women might be unwilling to teach in those districts. 
And, with a higher teacher turnover rate, thesrf districts bould be ex- 
pected to have a lower percentage of married teachers. 
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^Ho# Do- School Conditions 

and, Practicf ^ Influence i.namln)^? 

. Log*/ suggeits thit school condition! ind pricticei which impinge 
directly on the daily ' ive. of stirtents offer the raost promUe for 
improving eduction. Yey in many ways they are the hardest to atudy. 

The Observational Study attempted- to obtain data which would 
make ft possible to underatand better the importance of certain con-, 
ditions and practices. Some of the results are suninarlzed belbw: 

* • I. Teachers in high outlier schools made leas overt effort to 
maintain dais control, had less rigid student behavi^/but were more 
efticient in maintaining the level of contro^they apphred to want 
than were teachers in low outUer schools. 

2. Teachers in hi^jh outlier schools were rated aa warmer, more 
rcseonsiyt:. dnd placing more emphasis on cognitive development. 

3 More total ^ct ivi'ty takes' place in reading clajsea" in the 
high outliec schools. 

r.^Ho ^Vm^^^u^?!! ^" ^^^^ engage in more ailent ' ' 

^"ding children in low outlier schools engage in mor^ qral 

3. In grades one to three, teachers in high outlier schools 
gave more positive aAd less ne>iative reinforcement than did teachers 
In the low outlier schools. 

b. In grades four to six, teachers in low outlier sjchools gave 
more reinforcement. In general, however, they tended to use negative 
reinforcement more than the teachers in the highioutlier schools 

7. Pupils in 'the high outlitr schools were more enthusiastic.' 
and were better able to sustain attention. 

8. On selected items related to open education, the h^gh out- • 
iier schools appeared more often. ---^ 

9. Items on physiCc}l space and facilities generally did not 
differentiate between high and low outlier schools. 

♦tow Do the Attteudeq nf 

the Staff Relate to Learning? 



The attitudes of teachers are frequently cited as influences on 
the performance of students. For that reason a number of attitudes', 
perceptions, and expectations of teachers and other staff membera were 
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investlgated in the Observational Study. The findings include the 
following: 

I. Teacher* in high outlier schooli expected more children to 
* graduate froro high school, to go to college, to become good readers - 
and to become good citizens than did teachers in low outlier schools. 

2 Te-»chers in high performing schools saw their children «s 
more intelligent, *^etter behaved, more pleasant to teach, and their 
parents as more concerned. 

3. Teachers in high and low outlieV schools were aot' different 
tn the amounts of help they perceived as being available in handling 
problems. 

4. Reading teachers in the hi^h outlier schools gave more favor- 
able evaluations of the reading programs in their schools than did 
reading teachers i.i low outlier schools. The former rt rated the class- 
room teachers more favorably in using appropriate materials, extending 
reading into other areas, asking children to read with purpose, and 

* using Informal diagnosis. 

5. Principals in high outlier schools generally saw their 
personnel as more competent than Jid principals in low outlier schools. 

6. Principals In high- outlier schools saw themselves as having 
better rapport with teachers, parents, and pupils than did the prlncl- • 
pals in low outlier schools. However, principals In low outlier schools 
reported better rapport with the school board. 

SUMMARY OF FINDINGS 

Suniiuirizing the findings of a group of related studies presents 
a nun^er of difficulties. The volume of results resists a concise 
treatment. Attempting to discuss the results In simple terms may pro- 
duce ;nisleading conclusions. The ambiguities and contradictions between 
the findings of different studies may defy easy explanations. 

A particular problem in interpreting ctoss-sectlonal data, which 
these studies used, involves the extent to which variables can be 
inferred to cause the outcomes with which they are associated. It is 
part of the litany of research that "correlation does not imply causation. 
However, there is a human tendency to jump to conclusions about cause 
and effect Findings such aj> those presented here should be Interpreted 
with restraint and logic in order to avoid faulty conclusions. 

c 

, One area in which a logical analysis of the situation may avoid 

Incorrect conclusions has to do with the findings that special programs 
are frequently as*;ociate^ with low achievement. A hasty conclusion may 
b^ th4t the special program* ,ire ineffective or actually detrimental to 
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•tudent .chlevement. However, in,ny special programs-.f or the h-nHl 
We'l^^ed ^ ^^"^v.nt.ged, to cite two^n't'c^Xve been 

t^^ : "IrcU^^'" ' correUtlon''b:tween 

cne prevalence of tpecUl programs and achievement, rather than 
meaning that the program, have adversely affected ac^tevelnt ^ 
cates that the program. „e located where tLy Ire needed 

that a t^rTof'flnd\n'^s"fri: 'h^'h"' °' interpretation. It seem, 
useful Th! Ln from the three types o£ .tudWTmay be 

: 7 following .umwry shows which variables w/re a.soclat^H 
with students' achievement In reading and arithmetic iitir socuJ ! d 
economic factors were accounted for. ^ 

Factors Associated with High Achli>u>n.^„»- 

factors^ chlevement of students was associated with the following 

1. Higher teachers' salaries. 

2. \J^e of rooms for academic rather ^han vocational 

study . 

3. Open classrooms. 

5. Multi-age groupings. 

5. Higher levels of graduate training of teachers. 

Larger percent of Black teachers. 
7 Better control of classes but with less overt 

effort on the part of teachers to maintain - 

control. 

8. Less rigid student behavior. 

9. Greater teacher warmth and responsiveness. 
Greater emphasis by teachers on cognitive 

development . 
More total activity in reading classes. 
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12. More silent reading. 

13. Positive reinforcement of students by teachers. 
.More enthusiasm on the part of students 

15. Better ability on the part of students 

to sustain attention. 

16. Higher expectations on the part of teachers 

for th^lr students to become good readers 
and good citizens, to graduate from high 
school, and to go to college. 

17. Teachers' perceptions of their students as more 

Intelligent, better behaved, and more pleasant 
to teach and the students' parents as more 
Concerned. 

18. More favorable ratings by reading teachers pt the 

reading program in their schools. 
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19. More favor.ble rating, b, riding t«ch*r, of 
rlas.room teachers In u»lng appropriate 

extending reading Into other area,, 
a.klng children to read with purpose, and 
...Inn informal diagnosis. 
,0. Hlg r^itlngs by principals of the competence 
of personnel in their schools. 
Principal*' perceptions of a high °f 
rapport with teachers, parents, and pupils- 



21. 



.,.r.r, Assocl .rr- Achievement 

.ow achievement of students was associated with the following 

factors: 

I. Larger district enrollnicnt. 

7 Social program* for the handicapped. 

3* M^re pipil services, including guidance, social 

' * services, and attendance services. 

*rgl;";"rc:;r:f uncertified teachers in a 

district or school- 
rg:t^v:^:tSorc^«nt a students by teachers, 
mnclpals^ perceptions of. high level of 
rapport with the school board. 



7. 
8. 



.^K^m, Amb- r-- ■""^i""'"^'" "'^^ Achieveme nt 

number of variables showed amblgOou. relationships with 
student achievement. Among these were: 

\: pi";u;ir;xpendltures for Instruction, teachers. 




g.^rnrs Not A ».nrlatrd with Achievement 

several variables were found not to be associated with achieve- 
nent. Among these were: 

1. Attendance rate, once socioeconomic factors are 

cpnsldered. 

2 . Student nobility. 
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3. Various organizational and grouping arrangements, 

including clustering, continuous progress, depart- 
mentalization, differentiated staffing, dual progress 
plan, house plan, modular scheduling, non-graded, 
self-contained, and team teaching. 

4. Median age of teachers in a district. 

5. School facilities and space. 

0. Teachers' perceptions of the amount of help available • 
in handling problems. 



DISCUSSION 



In many respects, the findings of these studies seem to agree 
with other research conducted during the past decade. However, some 
very logical, long-held assumptions about the effects of certain system- 
wide administrative variables are not supported. None of the "expected" 
relationships with achievement were demonstrated for attendance, 
mobility, special compensatory programs and services, and gro&s'exoen- 
diture variables. 

o 

Mixed results were found for class size, school size, and inno- 
vative programs. Smaller classes in smaller schools and innovative 
programs appear to be found in both high and low outliers. 

The most encouraging findings related to teacher characteristics 
staff attitudes, and school practices. Result.s of the several studies 
Indicate that good teachers are the heart of the educational system, as 
conventional wisdom would suggest. Students seem more likely to achieve 
well where teachers are better trained, ncre often certified, higher 
paid anJ more likely to be tenured. The teachers in high performing 
schools have higher expectations for their students and more favorable 
perceptions of them; they app,;ar to be warmer, more supportive, and more 
responsive. This is accompanied by more enthusiastic students. Teachers 
In more :;uccessful schools. also appear to deemphasize strict control 
and lean toward more open education. These studtes do not inform us as 
to what "better" teachers do to bring about high ^hievement. nor do' ^ 
they explain why a warmer, more open environtoent is more conducive to 
learning. 

The relationships described are not necessarily causal. General- 
izations drawn from the^e studies should be considered in the light of 
other research and the decision maker*s experience and unique situation. 
These findings are offered as one more bit of information which can con- 
tribute to an understanding of educational processes. One conclusion 
seems apparent: Studying district-wide variables and school variables 
seems to t>e less rewarding than studying the teaching-learning interface. 
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